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nmi  bIhm  varying  demands  on  a  uaar  interface.  rssuMng  in  an 
aradn(|  variety  of  user  Msrfaos  appaaranoaa.  Applcadon  araaa  Include  tha  office  environment, 
•vigratmg  P^vonnwi,  iwhiri  snMunmsnit  vtvxm  msas  wwwonmin^  w  tonwi  wy- 
nosmg.  tvw  cnspwr  nonyw  to  ounra  Men  or  to  mwonwri  inis  nn  on  unr  nor* 

^LbA^BB.  V|kA  ^iAMa  ^B  adtMk  a^BA  AAdBB  LmAA^A^B  aA  B^BA  AiAHttA^ASlAA  A^B^A  JHlBBBAkBA  a^a  BflkakM 

non*  th§  mn  ■  s>  gws  to  rsaosr  «i  vnpniion  ov  mi  ssimmim  mo  onorsncM  or  sppt- 
canon  osmsnos  on  uwf  ranmi.  ins  rsnwnsig  cnspitrs  ov  ms  report  oonosntnm  on  mo 
sppNcation  area  of  software  engineering. 


3.1.  Office  Environment 

mo  ovnee  envuonmem  isvscs  10  vecnnoioQiee  eucn  u  ovnoe  vwonneion  systsfrw  (uiw)i  rnsnsgs* 
mart  information  systems  (MIS),  and  data  procaaaing  (DP).  In  this  appfcaBon  area,  usare  often 
haM  totad  knowdadna  of  comouter  avatems.  Sinoa  Ihara  la  a  larae  number  of  usare.  trainina 

I  ^Itw  BBB  B  AB  BpBB PP.  PWP.  vV B  W^^P  Vi  ^^P  ^Pb  i^^P  B  ^^P  B  ^P^^PP  PPi  PP^^Pi  PP  p  BB  PBB  ^BB  Bp 

users  is  a  major  concern.  In  OIS,  one  approach  to  Marfaoa  design  has  bssn  to  amulate  an 

An^BBABal  uddB  IkB  BBAABP  Ib  4pUaMbB  fa  aftl  Cab  BPPAiBBmIb  IIbb  Jab1<4bM  BBBBApMBtpB#  |»BB  IkBAB 

mmmwmm  ion  imcn  me  unc  le  wsvysssr  ^  ■  lej*  r  or  eutieei  me  oiedop  rnii8pnof  nn  oeen 
used  to  dsacrfba  word  procaaaing,  flng,  retrieving,  and  browsing  cspsMMss  that  are  ossentM  to 
orane  omce  wOvk. 

f\aAaBBBaA  bA^aIIbb  Ib  ^^bbA  bbbaba  ^^B  abb  ^dttjBA  MpbS  bbbbb^^bb  Ibbbb  itBbiaBBB  bJ  jIba  bbabIb 

Document  pvoouceon  le  ov  greet  concern  10  an  ovnoe  met  goneness  mgs  vosimss  or  Documents, 
eepedaly  reports  with  tablee,  llguvee,  Mbiogrephiss,  end  evooo  references.  Integration  of  docu* 

naad  to  move  documents  among  (Afferent  usare 
and  dMsrsnt  tools  such  as  documsnt  preparation  tools  and  alaclronlc  mafl  systems.  Another 
Issue  of  Intogralton  la  tha  Inclusion  of  speech  as  an  addMonal  communication  medfcim,  which  can 
be  In  toe  form  of  stored  and  retrieved  massages  [124|  or  speech  recognition  tods  [96). 

Management  totormatton  systems  require  support  tor  muMple  views  of  Information  through  bus!- 
ness  graphics.  Part  of  tha  viewing  operations  are  summaries  of  the  Information,  in  an  interactive 

ABBWftASM  bIabbbb  Sb  «Ka  U|1bPVVPbJ(ppji  |bb  bjpa  idAu  1a  bmppabSap!  Ib  Iba  bpphpjbjmbaIajI  ^ABBUBtAPt  babbbaa  An 

system,  ensngee  10  me  ssonnsDon  si  one  view  is  expecteo  m  oe  propegstea.  uommon  represen- 
xenons  vor  unoenying  ssormsoon,  eucn  ee  me  tspvosooumon  vor  spvceosnoev  oete  es  pvomoveo 
by  Mtoroaoft  (referred  to  as  SYLK  pep ,  alow  Information  exchanges  among  tools  not  necessarSy 
designed  together.  The  users  of  such  systems,  often  nonprogrammsrs,  should  not  need  to  know 
the  torminotogy  of  the  mechanisms  being  used  to  achieve  appropriate  views.  Based  on  the  type 
of  data  and  tha  desired  effect  (e.g.,  relative  comparison),  k  would  be  highly  useful  forth#  system 
to  provide  the  most  appropriate  and  affective  representation. 

umM  procsiimg  ons  wsn  mgs  vosjmm  ov  svonnraon.  omco  suen  Nivoiiiiauon  often  snocts 

g^BBABB  BP^d  |4^b1b  &BAA  bSbSb^AB  BbA  idpBJ^UIpBA  ^h.  g^BJ^B^BlBB^B  J  ^A  la^BBPBiBl^BflB  |b  IggAPBBABBd 

psopvs  sno  me*  wfss  ofBCiQfi  ^wwigent  cnsGvung  iof  oonsmency  m  ■woittsiioii  is  snpons^s.  i  ms 
aooflee  both  to  Marerthre  and  nonintaractive  data  entry  as  wel  as  data  retrieval  and  printing. 
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12.  Engln— ring  Environment 

The  engineering  environment  Includes  computer-aided  design  (CAD),  architecture,  and  mechan- 
toe  engineering.  In  Me  epptoeWon  area.  Momialton  to  beet  presented  graphically  to  show  prod¬ 
ucts  and  composed  components  that  potenttaBy  deoompoee  into  smaller  components.  Al  of  these 
oomponento  may  be  interconnected  In  ways  that  virtuaRy  are  impossible  to  describe  without  the 
aM  of  graphtoe.  Such  componanls  frequently  are  standardtoed  and  made  avalable  to  the  user 
onupi  onene  rcranee.  ueuaey,  mere  are  appecanon-speceic  constrains  regarding  component 
heerconnacttons.  These  conetoalnte  may  range  from  logical  consistency  to  layout  imitations. 

For  engineering  appfcattons  that  manlpulale  objects  in  three  dmensions,  the  use  of  three- 
dmanatonal  graphic  views  has  opened  new  avenues.  In  chemistry,  for  example,  visual  examina¬ 
tion  of  molecular  structures  has  yielded  new  insights.  Currently,  a  variety  of  techniques  are 
mrmwM  vor  rapnNivnB  wii  o>ntn>ionM  srapw  on  wowwosiona  screens,  ranging  trorn 
polygon  mMhM  to  staving  and  uss  of  color  and  M*nsly[4l). 

3.3.  RmI-TIitw  Environment 

The  reaHIme  environment  refers  to  appfcattons  where  time  to  a  critical  factor.  Two  obvious 
appfcaUon  areas  are  process  control  and  simulation,  (e.g.,  fight  simulation),  in  process  control, 
the  user  interface  to  expected  to  give  the  user  an  Impression  of  the  progress  of  an  ongoing  real 
world  process.  Thus,  a  large  variety  of  sensory  input  devices  wlhspedfic  appfcattons  may  have 
to  be  handed.  For  the  uaer  to  make  decisions  and  provide  feedback  to  the  appfcation,  infor¬ 
mation  shout  the  process  and  Is  changes  must  be  taptoasmad  property  to  be  communicated 
effectively.  Today,  large  dtoplays  Incorporating  graphtoe,  color,  animation,  and  audo  output  are 
commonplace. 

Simulation  appfcattons  try  to  model  a  real  world  environment  and  process  for  the  user,  in  some 
cases,  a  display  screen  can  sattofadoryty  present  a  colorful,  three-dmensional,  moving  world. 
Other  appfcattons  use  addttonai  meda,  often  phyaicaly  more  realsttc,  to  communicate  the  im¬ 
pressions  of  the  real  world.  Coordnated  control  over  these  devices  to  required  to  maintain  a 
consistent  view  of  the  simulated  Image  of  the  teal  world. 

3.4.  Mixed  Media  Environment 

The  mixed  media  environment  refers  to  systems  such  as  electronic  publishing,  electronic  music 
composition,  imaging,  and  speech  processing.  Electronic  publishing  pushes  technology  in  sev¬ 
eral  ways.  Digital  video  and  audo  daks  are  used  tor  information  dissemination  with  local  retrieval 
capabilities.  Onine  retrieval  capabOties  are  provided  to  a  large  population  through  videotext  and 
sfenlar  technologies,  which  are  being  expanded  to  provide  data  entry  capabilities.  Finally,  com¬ 
puterized  support  for  production  of  pubfcattons  to  provided. 

Electronic  musical  composWon  has  encouraged  new  techniques  and  technologies  for  entering 
information  and  for  visual  dtoptay  [7]-  Current  technologies  alow  the  composer  to  enter  a  score 
into  the  system,  synthesize  orchestral  components,  and  revise  the  score  where  necessary. 
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Standards  taetRute  (ANSI)  dtvolopod  a  terminal  standard  (AN8I  X3.64) 
,  which  was  reaRnd  in  draft  form  In  tha  DEC  VT100  tsrmlnal  wRh  minor 


HQSons  onaon— y,  wmcn  ton  a  ptossfoon  or  uspsy  oooss  si  ouson  10  so  pionson 
of  omuMion  of  ANSI  X3.64  or  DEC  VTIOO  functtona*y.  To  achiova  portSbRRy  of  appRcstions 
■cross  tha  ranoa  of  amaealno  ♦•wwSmi  tvcas.  the  Umx  nran  'ii  svstam  orovidas  a  vbtud  tarmi- 
nal  interface.  Tha  implementation  of  this  interface  usos  a  database  of  lonnlnal  suocilc  claiauei 
Istics  (tarmcap  [134]).  New  terminals  can  be  supported  by  addfeig  tha  appropriate  tarmcap  to  tha 


4.1  uL  Graphics  Output  Davlcaa 

Tha  dass  of  simple  graphics  terminals  represents  CRT  terminals  wRh  character  position  addree- 
saosay  ana  grapracs  cnaractar  sets,  cnaractsrs  si  tree  set  provne  rw  arswmo  eremenis  tna 
may  be  oompoeed  to  achieve  the  daoRod  effect  An  example  Is  tha  DEC  VT1Q2.  The  technique 

A^A  ^udafeftujh  SRba^A^^A^A  AA-AySlU.  _a^AbRb^^B  A  ^^B^AaA  I^a 

OS  QT^nC  CnSOJf  NV9  PfwVOSS  BfiOSO  QTHpmCS  CflpSDNSOSy  SSSCn  STS  UNO  iMOStljf  SI  l^pK* 
cation  arses  such  as  business  nr—aiam  .  tmb  tsch"^*1*  Is  also  used  on  manv  of  oarsonai  com- 
putare  The  ins  of  color  has  not  permeated  software  engineering  appRcstions  other  than  bust* 


Tha  dass  of  fuR  graphics  tarminala  Is  represented  by  two  harthrare  technologies:  vector  graphics, 
and  ftAmqp  graphics.  Vector  graphics  dominated  the  CAD  appRcation  area  for  a  long  time.  How¬ 
ever.  wRh  maturRv  and  Increased  dtaRal  orocessina  oower.  bfinso  araohics  became  the  more 

^^^B  S  V  W  B  ^A^y  ^^BB  ^AA  SB  A^AA  *.  BPB  f  A^AAB  B  ^A^^yA  .B.  BAA^^P  AbBbB^AAB  B  B^A  wB  ^Bb  B  ^AAB  BB 

prevalent  technotoov  R  was  mors  vmsla.  A  bRmao  dteolav  euooorts  indMduaRv  ad- 

a- BwvHVBr^aBBy  aa^^^pa^ba^^^^.  ^a  bbb^p  bb^bab^a  v  ^ab  ^^^bba^p^a  b  w  b  aa^abb^abaa  a^baa^a^bab  aba  bbbb  b  bbbpbbw  a^aa 

dreesable  pixels.  Xerox  Parc  became  a  forerunner  in  using  this  technology  in  a  personal  com¬ 
puter  workstation  for  ueee  other  than  graphics  appRcstions  [12].  The  bRmap  display  and  availa- 
bRRyof  Vw  mouse  as  a  pointing  devioe  introduced  a  range  of  new  interaction  techniques,  many  of 
which  were  explored  for  practical  um  in  the  SmaRtaRt  system  [48].  Tha  high  bandwidth  communi¬ 
cation  provided  a  means  to  represent  information  In  forms  closer  to  those  normaRy  used  within 
particular  domains.  Examptos  are  WYSIWYG  (what  you  see  is  what  you  get)  document  prepa- 
won  vypifni  ms  iynvxwc  oopn  rapwruDon  (loons),  ins  oociicaioQ  processing  powor  or 
fluch  i^i^sfts^ns  snsUss  s  hun—v^omp^sr  dstogus  thst  Is  more  user  friendly,  especially  for  non- 
oompmer  people.  The  resuis  of  tois  work  are  now  found  in  products  such  as  the  Apple  Macin¬ 
tosh  [130). 
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4.1.4.  Logical  Input  DovIom 


useful  insights.  A  eat  of  logleal  devices  has  been  defined  (88, 41]  and  is  supported  as  part  of  the 


a^ts^b  M^uM^Aa^h  uAkAbk*  amJ  88a^a^^m  t*akAlMMa  a^M^A  8a^a  AkmA^Ad 

fAw»  oon»  Dno  mo*  %pQ<no  01  um§  hmwi^ms  cm  do  oononoo  ooocy  oy 

b^mIa^af  MffSMM^^MAb^^A  amIaaSIosSa  Mi  Mk  O^AAA  ^AaIaa  aaIaaSIam  ^aa  ^a^a  ^a8m^ab8^^^m^8  faS^^^8aa^a  SaaL 
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this  group  are  baaed  largely  on  motor  ski  demands  in  hancflng  tho  devtoe.  An  analysis  of 
physical  and  *nn*fff*  davioaa  and  their  sunort  for  dffervnt  Interaction  teshnkmes  is  provided  bv 

^A*  — j  v^a  F*W  A  ^A^^A^A  ^AA  A*  A  FAi  A^**— ^A*  ^A  **  l^^^AAFB^AI^A^AAW  •  A^AA^A  ^B  ^^AMAFAPBA  ^AB  IFF  WVFBW  Av^F 

Fbloy,  Waloce,  and  Chan  [42],  and  Card,  Newel,  and  Moran  [26].  Foley  at  al.  deecdbe  a  sys- 

a^Mv^w4b%  aiMMiAiMW  8a^^m^^a8  baaa  A^^^b^A  4aaa  amf!  bfamidbAii  IaaIiaImiaa  a^maa  8a^a  ma^f^mfI  aa«8Ma  oAaaa 

vovnooc  ooimuio  dosog  on  wrocii  oovioM  Snu  Kiovocuon  i&cnmquM  con  do  maicnoa  ww  mi 
Interaction  tasks  to  be  performed.  Card  at  aL  determine  experimentsfy  the  effectiveness  of 
several  Input  devlcea  for  eedaln  types  of  operations.  Reaufts  show  that  keyboard  commands  are 
more  affacKva  for  short  dbtancs  posHoning,  whereas  a  mouse  to  more  effective  overall. 

4JL  Textual  Communication 

Martin  [75]  gives  a  somewhat  dated  but  stiff  vald  discussion  of  textual  dialogue  akamaHvaa.  The 
let  includes  Englsh-ianguago  techniques,  mnemonics,  prompting,  menu,  and  form  fflHng  tech- 
nlquaa.  Al  are  centered  around  a  oommand  language  and  afferent  ways  of  entering  oommand 


Command  languagee  are  the  means  a  communication,  not  only  for  the  system  command  inter¬ 
preter,  but  for  applications  as  woL  Command  languages  can  take  ths  form  a  natural  language, 
Imked  natural  language,  or  programming  language.  Natural  language  communication  would  be 
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m  aoms  sdtore  or  "capb&«ndsa  In 


pvwng  vvyvq  vnounii  ov  vtvtimmm  vmo* 


or  ■  mom 


oomptotton  (upon  raquaal  or 


ssedon43.l.). 

To  mm  to  too  preflbton*  ol  dtoarent  uaare,  oommand  Intorpreiare  support  savor*  typaa  of 
command  sntry.  Expsrtsnosd  uaaio  land  to  prslar  Msrfaoss  wth  short  quartos  and  taw 
Ireysaotoa,  relyfno  haarty  on  iwnamontoa.  tanclon  hays,  and  ismambarsd  asquanoss  of  opara- 


task  wth  quasdons  to  which  tasy  respond  by  typing  in  athartfw  ana 
horn  a  dtoptayad  msnu.  An  todMdual  uaar  can  ba  a  novtoa  wth  on* 


a  subsst  of  commands  and  a  novtoa  in 
augmontod  by  haquancy  of  uaa.  An  sxpr 
only  occastonaly  and,  thsrefore,  daaha 


rematodsr[l37|.  Tha  | 
oad  uaar  may  uaa  oort 
rant  support  for  thaaa 


mars  or  a  aaiacdon  charaotar 
tspploaUon  and,  at  tha  sama 
bnauaarcan  baansxpart  in 
proftotoncy  laval  of  a  uaar  is 
lain  taaturas  of  an  aooNcation 
>  taaturas  than  for  constantly 


Thasystsmmay  attsmpt  to  daduoa  too  laval  of  axpartisa  or  lot  tha  uaar  dadda.  Tima  sflscts  tha 
laval  of  sxpsrtlss;  in  othar  words,  nonuss  of  an  spplcation  may  regress  tha  uaar  from  axpart  to 
novtoa.  Tha  uaar  may  ba  fososd  Into  ona  dtabgus  stylo  baaad  on  tha  systanvdaduoad  laval  of 


.♦•v* 


by  tie 


•mbi fail  nnmmaidcaWn.  Thaoo  dtoptoga  team  Aral  taaMNi  India  tomi  of  ohamotor  CRT* 
augmented  wtoi «  flwpWo  olwwrtf  -t,  and  motor  gmphlo  tonwHato.  Rmh%,  mhmp  tor- 
mlMli  ha*e  pnimM  tie  mamet  of  oemputor  wadatotfene.  Togatnr  atoh  the  mHH%  of 
pawing doefcoa,  ftadgiyo hawa anoouragid new tonne of ueer Hwdon.  Thaoo  Inofcde 
town  totoip  Wia*.  WT8IWVQ  ayetoma.  Widow  eyotoma,  poWtog  at  Input  toohnlgae.  and  a 
dawofdWoguamaonanWm. 

dt  ito  on 

Wii«  MSpilj  rwimViy 

ftim  Whg  ayetoma  ham  baoome  qute  popular  a»  a  ueor  Irtondhr  torn  at  Wemcton.  even  on 
•hammer  tomvnato.  The  uoar  to  praaantod  Wh  a  torn  on  a  aoman  araa  dial  oonatota  of  ex¬ 
planatory  Homwion  aid  ftolda  to  which  data  can  ba  emend  and  modtoad.  Whan  a  fonn  to 


ayatom  may  nm  aome  coneialenoy  ohecke  to  eneure  Vie  oonect  type  of  dato.  maomacaaaa, 
conatralnto  on  tnvtoua  tinge  are  atoo  chocked. 

In  general,  tha  uaar  Indtoatoe  th*  a  loan  to  comptotod  by  toouing  an  expfc*  command,  in  aoma 
•yttome,  completion  of  tha  toil  fold  Indtoatoe  oomptotfon  of  Via  form,  m  addMon  to  being  uaad  in 
Via  data  onvy  araa  (04.,  aMna  maatvalon  oyatoma),  forma  ayetoma  hava  boon  uaad  aa  com¬ 
mand  Warfacaa  tor  programmara  to  hafp  raduca  tha  complexly  of  commwd  antry,  aa  In  tha 
Software  Productlvty  Fadtoy  by  BM. 


ftoecvnoo 


»•  '  I 


1  i  . .  i  Mi 


i  a  Nddsn  window,  Via  uaar  may  haws  to  psrtorm  on*  of  aavsial 


Moron  Hit 


as  on  «No  MaoMoah  [13QI  in 


Rbynamtaoa0hmanuaalBoaon,aBln  Andraw(ilO|; 

I  by  oommand,  aa  in  Emms  (181]; 

Mt  I  by  loon,  aa  In  S^iphiro  M  or  MtoroaoR  \Mndow«  [70). 


In  Vis 


►  wtovadappad  wtodowtng  tachnlqua,  windows  an  always  oompIsWy  vWbls.  Now  wln- 
oan  on*  ba  piaoad  In  pmvtous*  unuaad  sorsan  arsas.  As  a  rssuR,  ths  usar  may  havo  to 


My  uaad.  in  oadsr  to  mttm 


Mgli  lawal  of  actfrty  on  Vta  dhptay 


dMaretomnonovariapping  windows  in  that  thasorsan  area  is  always 

IwOriI  B  nBa  aBnwwi  BQBalQ  BFinCIOB^B  BF®  iBOHOO  BOQ^Of 

Mam  Dapandtog  on  Vis  ptaoamant  algorithm  usad,  this  rasuts  In  a 
dsplsy  scrssn,  mtfdng  I  dMIcuR  tor  thauaar  to  kaap  track  of  window 


dMouR  tar  Vis  ussr  to  arvangs  asst  of  windows 


nt  of  now  windows  [110, 70].  TTtay  oftan  maka  I 

Hi  I  CVnBn  wly.  rtw  W1IUUW  lyiiiml  MOW  m  m 


«ah  (M  the  tmout  oan  to  automattoahv  set  ud. 


tfMt  them  as  on*  working  context,  sknlar  to  the 
user  to  ««t  up  several  contexts,  not  tel  of  them 

MNKn  OMWVPl  tflpn.  Arl  WnllM  10  WwKJOW 

totoh^AfdM  tA  In^M  AiAu^mlMiM  fOto  *e*l  Ujuu^.^. 

wrtDl  I  wnOOl  WO  WDMWQwl  [wj  wj*  HOWlVlfi 

CSadSf  (130)  provides  a  Qmnl  conospt  of  windows 


Ml  pop-up  sorol  bora,  thumbing  facMes, 
I  or  pointing  dovioo.  Users  may  to  tote  to 
i  (dsiete  end  insert)  operations.  For 
can  copy  a  piece  of  Usp  code  received 

_ _ ,  _  and  execute  K[12Q|.  Some  systems,  e.g.. 

Mown  MwviiwKrvit  iuhw  wwyv  o  mvnwo  ivki  ana  grapnm  as  ww  as  pwi 

“  “*  ,  must  use  a  oommon  representation 


in  the  Xerox  MarLtap 
throupt  electronic  mil  into  the 


4AX  Twro-Otmanalonal  Input 

TW»  SUMkBlMMi^iMhifeA  AA^JflMkB  A^k  MpAtf  Aa  Ak  MMlkfli  Mk^k^kSeSAkAk  AkJA  SAk^k  AA^AAm  *-  -1  — 

vna  wo-camnwonv  (npay  n  mo  any  a  rwar  a  a  vanoovn  poaaion  on  ma  aciaan  (va 
a^aaro  ^wiaiiBno  a  ^wa  waoa  pav^s  nai  ways  10  oonvnunaaat  a  vanay  oi  pnypeni 
Input  devices  exist  Devtoe-indopendont  graphics  packagse  such  as  OKS  [61],  and  works  such 
as  Foley  and  van  Dam  [41]  and  Newman  and  Sproul  (88)  use  logical  Input  devices  to  achieve 
physical  dlvfoe  independence.  The  articles  by  Foley  and  van  Dam  [41]  and  Newman  and 
Sproul  pqdteousa  the  ralatlonaHp  of  logical  toput  devices  to  physical  Input  devices  as  wel  as  to 

le^^AAM^AM^keh  ^jlMlikiAA  |aaa  aIaa  aa j^Iaa  A  A  e\ 

oaaai  nawinn  wCTwqwi  (tvt  avo  Mcoon  4.i,4j. 

One  of  the  more  popular  pointing  devices  is  the  mouse,  which  can  have  a  number  of  buttons 
oommonly  ranging  from  one  to  tour.  Mouse  buttons  can  to  considered  special  function  keys  that 
suggest  execution  of  functions  related  to  the  mouss/oursor  posMoa  As  can  to  seen  from  exist¬ 
ing  systems,  ftmtoons  performed  by  a  mouse  wth  muMpte  keys  are  often  simulated  by  a  one- 
button  mouse  or  Keyboard  have.  The  folowlnfl  set  of  exemoles  *■«*«>•  this  point. 

Otrsctlon  can  to  Indtoatedwlh  a  tour-button  mouse  where  toe  buttons  act  as  cursor  keys  (Perq), 
Wh  mouse  movement  (MacintoetVFiogger),  or  wkh  cursor  keys  on  the  keyboard.  Sctodngdrec- 
tlon  can  to  todcated  wkh  one  of  two  mouse  buttons  (Petq/Psppei),  by  cNddng  In  the  appropriate 
area  in  the  ecrofl  bar  (Xerox  Star),  by  function  keys,  or  by  menu.  Selection  of  more  than  one  Hem 
oan  to  performed  by  hoidtog  a  mouse  button  down.  A  selection  can  to  extended  by  using  an 
aha  mate  mouse  button  (Xerox  Star)  or  by  using  a  key  In  addMon  to  the  mouse  button 
(Macintosh).  An  •open"  or  "activate*  operation  on  an  object  can  to  performed  by  double  clcking 
(quickly  pushing  the  mouse  button  twice  (Macintosh,  Interleaf)),  by  menu  selection  (Macintosh, 
Interleaf),  or  by  object  selection  and  invocation  of  special  (unction  keys  (Xerox  Star).  As  can  be 
seen  from  tf«e  examples,  in  many  oases  context  Information  such  as  location  or  time  (as  in  button 


hold,  took,  and  double  took)  is  used  to  enable  one-button  mouse  operations.  The  different 


(Mwii  malar  idh  (amount  at  motion  and  usa  of  ona  vs.  two 

a^pp^^^p^^a  p^b  a  a  ■  ^pbp  a  ^w^^Pas  s^^p  a  a  a  ^b^p  ^ap  a^^p  aaa  w  ^p#  a  wwaa 


(Man  Macaefiona  la  ha  —"■m  bv  tha  keyboard  as  wel 

assisiBa^es  ^e^^s^^ea^w^e^a^u^^w  ^ap  a^^^F  ee^y  ^e^ae  e^WF^ya^a^^^^aap  ^w^p  ep^p^s* 

and  ftirtTwri  an  >»sii|4ai  of  taxt  adftors  whan  cursor  motion 
inouaa.  In  the  case  of  Pepper,  the  functtone  are  bound 
mouaa  also  esrvee  tha  ksys  assigned 
Lad  varaua  rioM  handadnaaa  naada  to  ba  conaidand  in  such 

ep^SP^w  w  p^s^^^a  B^war^awi^a^w^^^^s^^p  e^^^^epa^aa  ^ae  a^^w  a^Bpe^p^^B^^^paapap  ap  ^apmv 

Stow  oommands  to  be  invoked  by  function  key  or  by  menu. 


Ttra  apprafan  cursor  ce*  be  bound  to  tha  mouaa  location  or  decoupled  from  tha  mouaa  (for 
example,  for  text  anby).  In  tha  Mat  caaa,  the  application  cursor  poaMoo  (in  general,  representing 
n  fffxn  poNOVQ  s  cnano9P  oy  wnpii  fnou$9  inovinwi  Aocwira  moun  vnovwnsni 

IVMpMOW  B9  ^fpKwOn  GUfNT«  Hi  HlH  8HGO11Q  C8B0|  u9  flppHCHDOn  CMe90C  19  0OH  9HH9CH90  Dy 
■OOWw  mOU99  VnOVlvVMfVs  DIH  Hl(J8t09  plHCHQ  D y  HXpiCH  OpHUHOUf  H«Q*f  pU9ninQ  9  >001199 

|u  Jbui  ||^  uXy^BB  J||kB  yUllfiAUI  aBBBLtlaMP  ||kb  BugB||l  Lmb^I  BUk  1^  jI^^bbh|LbbpI 

DUOOn.  QmWnjfi  109  9G999  H91009  (uiv  WtnQQW  fBCHMllQ  109  ftHTHin  HpU)  Ctfi  DH  0919009190 

dksctfy  bv  the  mouaa  h>"*11"-  — -  potentially  caueina  unintentionai  window  activationa  —  or  bv 

BbbMbM  ^B^B  ^B  BPBflIBB  bMbIp  PbP  bI^bMb^I^BB  EiB  B^b^B  BidBBB  BdKB^BB^S^B  ABajlIpB  |SbB 

9Xp9C9  OpOfjHOn  9UCn  90  9  1091199  CMX  rOr  app9C9D0n9  In  WilCn  Wm  CUr90f  oRlCHy  uoCKt  W 

mouse  location  (slmuMlon  programs),  I  may  ba  desirable  to  have  tha  apptcatlon  cursor  rscog- 

b^^b  |S^B  ^B^B  B  BB^t  I^BIiB^BB,  |b  «L|b  ^^BBB  b  bbBPbIIbb  ^b  BB^BBBBBJ  &b 

nS9  109  9900091  Boundary  90  9  0910  00UTI09iyp  Ml  WI89  C999»  9  9P9CW  0p9ran0Vl  9  n909999ry  10 


Mouae-ftwohad  oparadona  are  often  lenine  to  context  and  location,  which  ptovldaa  more 

H9XK9Hy.  aMPHIH  0Qm9  OH  9  91009  nl9y  P900K  O0T9f9f9  00090900  99H9  10  D9  C9990  0y  pOjp 

ud  menu.  For  axamole.  In  Andraw  ftlOf  Pie  mouse  t^wiaw  in  a  p*»ipp  haadar  ralars  to  window 

b^^bp  np®  ■  b^^p  p ^bb  pbbwpp^^  a  >  b  bpb  ^p^bp  ppn^p^^^^p  ^b^^p^^bbb^b^^p  >  j^pb  ^pp  ^^p^b  ^b^^pp^^b  b  ^^^^bi^bp^^pb  p^ppp^ppp  ^^>p  ^^p^B^a^^BP^^p 

eyetam  commands,  wharaaa  appBcaBon  commands  am  avsMbla  whla  tha  ouraor  la  locatad 
ppaten  a  window.  Ddaul  loona  tor  tha  mouaa  cursor  okra  tha  uaar  faadiack  about  conlaxt 
changes.  The  avaSefaWy  of  a  menu  may  depend  on  the  stale  ofthteppacaUon(e^.eis  saving 
operations  In  Interleaf  Mflh.  flu  Ilia  Mai  ISnali  a  iMlaiaiS  mjuiuwjIi  la  Hfcan  for  LDnwnand  maim 

wyrwPiHwrp^  ai  a^^apa^^a  ^pnrj^p  apppp  pp^a  rw^a^p^a^wp  w  p^^awrwp w  eaMWPreaw r  ^p^^^wp  ppw  a^appp^pp^^pw^pw  pi^sp apwi 

Applcaflon  oommands  cen  be  found  in  e  global  menu  set,  which  resides  et  e  Used  location  on  the 


ooattcnlno  aoeradona  such  ee  horizontal.  vartioaL  and  canawfiaa  acroMno.  and  pUptWiib  oosMon- 

kna  bjbbbbSIbbb  m  «bSp  aa  Sla  tnablaa  iBBlalli^n  Aa  a  iBanAlaa  t«JlAata  a  ^^a*  *BaaaSlt  bIbIbb  bb«S 

»*8  Op9f99O09  919191  99  WnunQ  \pOV9nQ  10  9  9p9CMC  ©09000  919010  9  OOGUnMv^i  9Z9Q|  9flQ 

modon  of  wlndoppe  are  rsatead  by  maWng  use  of  location  context. 

4A4.  Mtnu  Mbchanlgma 

A  menu  aflowt  the  user  to  select  from  a  standard  set  of  ob)ecta  or  operations.  A  variety  of  user 
iniarfaoe  toolhfiffee  end  user  Inlerfifie  mananamant  pvatama  orovide  menu  mechanisms  with 

aawipaarw  wpwwpppa^ra  swap  a^^wr  a^roiipppw  pvwp^^ywi^sivp  uy vpwv p yawwapw  pp^pp ppp  pppwppp^bpi^ppp^  pwibpp 

dRevenldegraesoffuncdonaMypt4,49,9,57, 60,112,85). 

Menus  cen  be  vlsfcls  permanently  or  temporarfy.  Permanent  menu*  tend  to  be  represented  as  a 
set  of  buttons,  poasMy  representing  the  function  eymboicaSy.  Temporary  menus  ueualy  are 
organized  as  a  row  of  textualy  labeled  entries.  They  become  vMbie  on  top  of  other  windows 
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under  user  or  appic^ion  control  (referred  to  as  pop-up  of  putt-down).  They  can  appear  in  a  fixed 
place,  or  near  Ihe  mouee  cursor  location.  Menu  entries  cm  be  text  or  icons  or  both  (afl  these 
forms  are  found  on  Macintosh).  The  size  of  the  menu  window  can  adjust  to  ths  number  of  entries 
or  can  be  tored.  There  can  be  an  upper  In*  on  the  number  of  entries,  or  the  menu  content  can 
be  ecroled  or  paged.  The  set  of  menus  and  their  contents  may  be  frozen,  or  they  can  change  at 
runtime.  A  subset  of  the  entries  may  be  active  at  any  time. 

Temporary  menus  are  catisd  up  and  selections  are  made  in  one  of  several  ways.  Some  systems 
dtopiay  the  menu  when  a  mouse  button  is  pressed  Whtie  the  mouse  button  is  held,  the  mouse  is 
tracked  for  menu  entry  selection  by  Mqhtiphting.  Releasing  the  mouse  button  causes  the  cur¬ 
rently  highlighted  entry  to  be  selected  Release  outside  the  menu  window  aborts  die  menu  selec¬ 
tion.  In  the  latter  case,  the  invoker  of  the  menu  function  must  be  able  to  handle  this  exceptional 
condtion,  i.e.,  no  selection.  Other  systems  dtepiay  a  menu  upon  mouse  clck  or  function  key 
invocation.  The  user  can  then  select  an  entry  through  a  separate  clck  operation.  Invoking  a 
function  key  is  usualy  for  menus  wkhmuMple  selections.  Multiple  selections  may  be  indicated  by 
cflcking  multiple  entries  and  terminating  the  selection  process  by  an  expRck  "dose  menu"  com¬ 
mand  or  by  repeated^  cattng  up  the  menu  and  addktg  to  the  selection.  Selected  entries  are 
indicated  by  a  check  mark  or  by  Mghlghtlng.  Sometimes  the  eelection  state  is  remembered  be¬ 
tween  menu  cal-upe.  It  provides  a  context  for  extension  of  selections  or  defauR  selection  based 
on  previous  actions.  In  such  a  case  entries  may  have  to  be  unselected  expRcRty,  or  a  reset 
operation  is  provided  For  menus  with  state  memory,  single  entry  selection  can  be  enforced 
through  so-caled  rmMo  button  behavior,  which  means  that  selection  of  one  entry  removes  a 
previous  selection. 

Menu  entries  may  be  grouped  and  organized  Into  menu  hierarchies.  Al  levels  of  a  hierarchy  may 
be  vWbte  at  once,  or  they  may  have  to  be  navigated  wRh  only  the  selected  path  visible.  In  some 
systems,  a  special  entry  In  the  top  level  menu  permit  the  previous  selection  to  be  repeated 
[110, 60].  This  may  be  carried  out  by  Immedtete  reselection  of  the  final  entry  or  by  time  delayed 
waldng  through  the  menu  hierarchy. 

Menu  selection  may  be  in  response  to  a  question  or  a  message.  The  text  may  appear  separately 
or  together  wth  the  menu  entries  in  a  so-caled  dUtiogue  or  alert  window  [5]  —  also  referred  to  as 
prompters  and  confirmers  [48].  For  ranges  of  numerical  values,  selection  can  take  a  different 
form  such  that  the  set  of  dscrete  values  is  mapped  onto  dials  of  various  shapes  and  forms.  The 
user  can  seiect/set  a  value  by  absolute  reference  or  change  I  relative  to  a  given  value.  The  user 
may  interact  wth  a  window  that  combines  the  above  selection  mechanisms  with  fields  for  text 
entry. 

Because  the  muWpRdty  of  alternatives  is  overwhelming,  there  is  a  need  to  classify  the  tech¬ 
niques.  The  techniques  have  to  be  appropriately  mapped  to  the  interaction  tasks  to  be  performed 
by  the  user.  The  Descartes  Project  [113]  examined  the  menu  as  an  sxample  technique  and  used 
sutatole  abstractions  for  Interactive  communication  to  determine  the  role  of  menus  in  dtalogues. 
In  the  process,  a  design  space  was  determined  that  explains  the  variety  in  menus.  A  more  gen¬ 
eral  treatment  of  interaction  tasks  and  techniques  is  presented  in  Foley,  Wallace,  and  Chan  [42]. 
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"anted  daasMcarfon  rather  than  the  system-oriented  classVl- 
Ion  tasks  art  prlmMvs  action  unit  performed  by  users.  They 
Inleracfen  tasks  and  oonptsts  interaction  dWoguw  such  as 
to,  e-g..  the  Macintosh  tooldt  (51  and  usar  interface  manage- 


FWey  at  aL  (42)  dsflne  Ac  types  of  fetteratfon  tasks: 


1.  ^alaoMon  —choosing  bom  a  set  of  sMsmakni,  a^  by  msnu  or  abbreviated  name 
MAW 


Z.  FaaMantag — Mealing  a  loonton,  often  kt  the  context  of  a  oommand.  to  plaoa  m 


9.  Orienting — apeeiying  an  endys  dbectlon,  04,  by  rotation; 

4.  Path— a  sequence  of  poaMona  and  orfantaliona  ordered  over  flme; 

5.  duanNying —  spscftyfng  a  value  (uauoly  in  a  gken  range)  through  or  Mention  on 


«.  Tart  —  entry  can  ba  carried  out  by  typing,  or  by  character  or  word  aalsetion  from  a 
msnu. 


m  addHon,  they  tdsntty  lour  types  or  oontral  tasks.  La.,  taaks  dkactly  modftytng  dtaptaysd  obisets: 
1.  SMeNho  ~  alastic  ischniquas  such  as  thoaa  uaad  to  oonstmet  Inaa.  ractangtoo. 


2.  Sketching — freehand  drawing  wMi  tha  toeator  acting  as  a  brush  or  pan; 


3.  Manipulating  —  moving  objects,  e.g,  placing  thorn  In  now  posNona  on  tha  screen 
(also  known  as  (tagging)  or  acaitog  an  object; 


4.  Shaping— changtog  the  shape  of  a  smoofti  curved  Ine  or  surface. 


Constraints  can  beappled  to  boft  interaction  and  control  tasks.  One  common  form  of  constraint 
appOsd  to  orientation,  path,  and  stretch  tasks  reouRs  In  enforcing  horizontaftertlcal  kies  or 
circles.  Qrtd  and  gravty  are  two  forma  of  constraints  that  affect  poeHoning  and  other  tasks  that 


Each  of  the 
menus  or  0 

aKsmolB  a  1 


la  carried  out  by  Interaction  teohniquos.  Examples  of  selection  techniques  are 
nd  typing.  For  each  technique,  parameters  provide  further  dstinctlons.  For 
la  a  aalscllon  technique.  Menus  can  be  hierarchical,  entries  can  be  ordered,  or 
can  be  provided  for  menus.  Menus  can  pop  up  or  be  puled  down;  they  can 
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44.  Graphical  Communication 

Hit  August  1906  Issue  of  ZEBE  Computer  was  dedtoated  to  visual  programming,  l.e.,  the  use  of 
grapMoa  In  tha  context  of  programming.  Grafton  and  tohkawa  pi]  define  the  term  and  survey 
work  in  the  area.  A  second  article  bv  Raeder  1981  surveys  current  nfflPhtowl  oroarammina  tech* 
njqueo  and  oomparee  vartoue  graphical  programming  systems. 


juUBBkaaiBA  P^^^MMnfa  «i  bMBPBBMBBB mbIbbJIbIB  uJ^bA  ibB  j^bPBbI^BB  ■ A  ItBBBB.  IbbmIb 

myw  O^fWIS  pinTK  iWw  wn»  Cl  MinnafKWn,  Of  MUCn  Wi  OOfmQGr  vOUT  lypw.  SOOfWG, 

^A BBmA^BbI  •  JbpBJ^B  buMmAIbB^  BBBBBBBBBBfBB  BJB^  BMlBB^afBB  Lb  bbUMaa  A^B  -  -  — - - aB^a  BBBBBM  amlB^AB 

gnfmmSm  yyiCgprPflnwif^  «na  WMon.  in  looiion,  no  user  m®y  oommunicaia 

to  tha  systam  tasdualy  *Mto  tha  system  producaa  graphical  rapraaamaUona.  In  this  case,  tha 
la  4^  tsosaajpisoaiQpos  torliti  ai^aiotii^so^  ^^n  ie  PIC,  a  lan¬ 

guage  for  drawing  simple  flguroe  [87].  Drawings  spedfled  in  PIC  are  fed  through  a  document 
processor  (Troff)  and  can  be  viewed  as  printout  or  on  a  workstation  screen.  Other  examples  are 


Iconics  refers  to  the  factty  for  attaching  a  symboic  represent atton  to  an  object  or  entity.  Xerox 
Star  and  MadnlMh  are  oood  examnlae  of  an  uaar  interface,  leone  mav  be  defined  for 

BrlWP  Bi  ^PP  P^BBB^B  ^BP^^P  B  ^B^B  ^^P  P^B^B^BP  ^PBSB  B  B^^B  ^B^B  B  BBPP  ^ap^^BP  SB  VSR^W  B  ^^^P  P  ^WT  B^^P  B^^PP  W  ^VBb  PBPP 

each  ohioct  or  for  fi mutes  of  obiecte.  In  the  latter  case,  a  name  mav  be  included  hi  the  icon  to 

PPB^PB  1  BPBPWBPBPB  BPB  ■  — P*  BP^BB^^BPBPBP  BPB  WB  PPPW  ^WBw«  B^B^W,  BB  B^BBPB^^B  BB^WBY  BP^B  aa^BHPwlB  BB  BPPw  ^BBBPB  B  BBP 

dbtinguish  members  of  a  class.  Icons  ad  much  ttca  menu  entries  In  toat  they  are  seledsble. 

Daimu  MkABte .  a>aatePBM  AtMBlJBal  ^a^a  aMkaBaaiBSallaBk  ^^S  efiA^A^a  ,^a^^44aba  as  S  Sa  aIa^Ba  a^a%  SAULa.4a^ 

BrOnio  menu  syssems  suppon  on  iconic  r^pfosofUBon  w  hiOw  onuiOv  (o^*,  MOGwOOinj.  wfion 

created,  icons  are  placed  by  the  system  or  by  the  user.  Users  may  place  toons  in  arMrery  loca¬ 
tions  (Macintosh),  or  into  a  predefined  grid  (MS-WM).  Some  systems  provide  an  operation  that 
organizes  icons  into  grids  and  elminatee  gaps  (e.g.,  the  Macintosh  dean-up  operation). 

Movement  or  icons  may  nave  onerent  semanac  meawngs  oepetxang  on  tne  dwtoftanon  wesson. 
Within  certain  regions,  I  may  Just  relocale  the  physical  dspiay  objsd.  in  other  cases,  it  may 
move  or  copy  the  underlying  object  On  the  Macintosh,  for  example,  fltos  are  moved  by  moving 
an  icon  from  one  folder  (dkectory)  to  another,  but  fltos  are  copied  by  moving  the  icon  to  a  folder 
on  a  different  disk.  Moving  an  toon  on  top  of  a  trashcan  toon  delates  toe  underlying  objed 
(Macintosh),  and  moving  an  icon  onto  a  printer  icon  prints  the  underlying  file  object.  Printing  on 
the  Macintosh  and  deleting  on  the  Xerox  Star  are  executed  by  ether  function  key  or  menu. 

Some  systems  permit  the  user  to  define  iconic  symbols  and  attach  them  to  existing  or  user- 
created  objects.  Such  systems  are  user  extenstoto.  An  interesting  example  of  an  extensible  appl- 
cation  wlh  an  iconic  user  interface  is  Fleviston,  a  visual  database,  on  the  Macintosh  [83]. 

Iconic  rnnrsasnti  tms  are  attractive  her  wise  thsv  oermt  the  aooKcation  tmotomantor  to  erased 

•Bwpimp  b w^pv w^raB n^hvbpv re  w  BpBPB^tePWw  w *BPy  ppwvrite  niw  sba^^whwpvv  n.^rreiireiwBP*  bbp  ppvwww 

to  the  user  a  view  that  more  closely  reflects  his  mental  model  of  the  appfleatton.  However,  I  is 
dWtoutt  to  appropriately  apply  this  technique  and  generate  a  semanttoaly  consistent  user  toter- 
face  behavior.  This  is  Hustrated  wth  an  example  on  the  Xerox  Star.  Icons  (and  the  objects  they 
represent)  can  be  moved  and  copied.  Moving  means  removal  from  the  source  location.  However, 
If  the  destination  Is  the  printer,  the  move  operation  Is  translated  into  a  copy  operation  to  avoid  an 
unintentional  deletion  of  the  icon  (objed).  Such  action  makes  the  user  insecure  because  he  is  not 
sure  when  such  a  reinterpretation  takes  place.  Alternatively,  the  system  could  prohbft  the  opera¬ 
tion  or  ask  for  confirmation  as  a  protective  measure.  Graphic  functionality  of  a  display  permits 
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graphical  viewing  of  various  d— mi  of  Intormston.  One  dees  that  can  be  viewed  effectively  is 
quanttaive  Information  such  as  buefneea  graphics.  Such  representations  for  numerical  data, 
inducing  bar  charts,  pie  charts,  and  histograms,  are  highly  effective  for  summarizing  large 
amounts  of  dtta  and  ttwlr  retttons. 


Graphical  representations  of  structures  ate  also  effective  for  describing  the  complex  interdspen- 
dendes  In  ths  staiduto  and  at  dWetent  levs*  of  abstractor!.  Graphical  representations  also  can 


data  structures,  data  flow,  and  control  flow  can  be  represented  graphicaly 
{122, 107, 120, 86, 117].  To  date,  a  number  of  Interactive  systems  have  been  butt  to  support 
graphical  viewing  and  navigation  through  program  cods  and  documentation  in  software  devei- 
opment  (PV[16J,  PECAN  [104]),  program  data  structures  in  debugging  [128, 84],  and  directory 
structures  (Perq[ligp. 

Graphical  programming  uses  the  dMtty  to  arrange  graphical  symbols  and  interconnect  them  to 
create  a  graphical  repreesnteHon,  or  view,  of  a  program.  Such  a  graphical  representation  is 
expected  to  have  wel  defined  syntax  and  semantics  and  is  translated  into  other  representations. 
In  this  reaped,  graphical  programming  is  sknlar  to  computer-aided  design  (CAD).  In  an  inter¬ 
active  system,  the  user  is  Informed  of  inconsistencies  during  entry  of  the  structures.  The  inter¬ 
active  approach  gives  the  user  more  Immedate  feedback  and  can  aid  in  determining  appropriate 
connections  (e.g.,  PECAN  [104]).  In  more  common  systems,  the  user  enters  the  structures  with  a 
graphics  edtor,  and  the  consistency  of  the  structures  is  checked  by  invoking  a  separate  tool  (e.g., 
DesigrtAid  [30]  for  data  flow  dagrams).  The  capabttbes  of  a  graphical  programming  editor  in- 
dude  basic  operations  as  wel  as  elastic  connections,  layout  cleanup  of  objects  and  connections, 
grouping  of  objects,  and  hierarchical  organization  of  the  graphic  structure. 

Animation  is  the  abttty  to  show  changes  over  time.  Graphical  animation  can  show  such  progress 
over  rime  effectively.  One  way  Is  to  visualze  the  time  axis  and  generate  a  time  plot  of  values. 
Another  way  is  to  animate  the  change  Ksetf  In  slow  motion  such  that  the  user's  eye  can  perceinve 
the  change.  Animation  can  be  used  effectively  to  Bustrate  the  actions  of  a  program,  thus  helping 
the  user  to  understand  and  leaf  the  actions  of  an  algorithm,  and  to  recognize  bugs  and  logical 
inconsistencies  by  demonstrating  changes  to  data  structures  or  progress  in  control  flow  [16, 71]. 
Animation  has  also  been  used  to  morttor  performance,  such  as  network  traffic  and  program 
execution  profles.  Finally,  animation  is  a  tool  for  visualy  displaying  the  execution  of  simulation 
models. 


Graphical  simulation  has  been  common  practice  in  the  artificiaJ  inteigence  community  for  job- 
shop  type  simulations  and  is  avalabie  in  product  form  (e.g.,  Knowledge  Engineering  Environment 
(KEE)  from  Intettcorp,  Knowledge  Craft  from  Carnegie  Group  Inc.).  Though  graphical  simulation 
Is  not  yet  an  estaMshed  tool  in  software  engineering,  it  is  starting  to  raise  interest  in  areas  such 
as  modeing  telecommunication  applications  [78].  Functions  expected  in  the  user  interface  for  an 
animation  tool  include  controls  Be*  those  on  video  recorders  for  dteplay  speed,  and  reverse 
animation  where  posable.  Other  functions  wi  permit  the  user  to  initialize  the 
anknaton/SImulaton  environment  and  change  it  while  animation  is  suspended. 
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oonoepls In  dtotogue  management  are  becoming  avaRable.  They  am  the 
dsskijp  ffMffptior.  mtrpto&ng,  the  notion  of  bnwt/ng,  the  aWfity  to  provide  multiple  vims,  and 
anntopSMitod^  M  •  mdevorir  of 


introduced  by  researchers  at  Xerox  Parc  and  appeared  for  the  first 

The  acreen  represents  a  desk  on  which 


common  to  the  office  environment  shown  as  icons.  They 
tactade  kt»x  and  outbox  for  etoctmnto  mal,  ne  drawers  and  folders  tor  (led  documents,  printers, 
etc.  Objects  can  ba  opened,  Le.,  their  contents  examined,  in  overlapping  windows.  Thus,  a 
i  f  wortdng  environment  for  a  particular  kind  of  user.  A  user  can  potentfaHy  have 
one  desktop  defined  and  aRemate  between  them,  which  is  similar  to  the  way  people 


His  interesting  to  note  that  systems  with  an  "advanced”  user  interface  (systems  that  make  use  of 
windows,  menu,  icons,  and  forms)  look  the  user  into  one  form  of  interaction,  such  as  the  Xerox 
Star  system{112].  This  helps  to  moke  ffto  system  learnabto,  but  Imks  effective  use  of  R  by  some 
user  groups  such  as  keyboard-oriented  users.  Some  syatome  (Apple  Macintosh)  [139]  provide 
keyboard  aRsmallvts tor  some  of  the  frequently  executed  commands.  These  are  not  easily 
extended  by  users. Other  capabRRIes  of  command  languages,  such  as  background  execution  of 
programe  or  Interactive  deffnfifonof  command  eequenoes  as  new  commands,  often  seem  to  be 
ignored  for  the  sake  of  this  "new*  technology.  Such  features  are  eometimes  desks  ^  lor  sophis¬ 
ticated  users  to  enhvce  the  user  interface  of  an  applcatioa 


User  taRoring  or  profiHng  is  the  abflly  of  the  user  to  adapt  the  user  interface  of  applications  to  his 
needs.  This  means  that  the  user  interface  and  the  appRcafions  have  parameters  that  the  user  can 
set  to  determine  certain  aspects  of  the  interaction.  Examples  of  parameters  are  keybindtog  of 
functions,  rflferent  modes  IndfcaHng  forms  of  mouse  tracking  or  edUng  (e.g.,  expficR  vs.  impRcR 
insert  mode  in  Emacs),  or  preference  for  menu  driven  Interaction  vs.  function  key  interaction. 
Some  capabRRiea  enabled  by  parameters  can  be  used  to  simulate  user  interface  concepts  that 
may  not  be  supported  dkectly.  For  example,  the  abtty  to  specify  a  script  indtoating  a  set  of 
windows  and  appBcalions  to  be  started  when  logging  Into  the  system  can  have  the  effect  of  a 
desktop  to  a  Imked  degree. 


Ueer  talortng  radsee  several  questions,  for  example,  what  is  the  desired  degree  of  tailoring?  The 
Xerox  Star  system  supports  only  minimal  variations  On  the  other  hand,  the  Unix  system  pro¬ 
vides  a  large  degree  of  freedom.  TaRorlng  is  in  potential  conflict  with  uniformity  and  portabHty  of 
users.  R  permRs  one  user  to  tailor  the  environment  to  his  needs,  but  makes  R  hard  to  interact  with 
other  users  about  the  use  of  the  system  and  provide  general  help.  TaRoring  is  used  to  overcome 
shortcomings  (as  perceived  by  the  ueer)  in  the  appRcation’s  user  interface.  Is  this  the  right 
solution  to  overcome  these  problems? 


The  notton  of  browsing  has  been  promoted  by  a  variety  of  groups  (e.g.,  ZOO  information  network 
(43,44),  document  browsing  [140],  the  Magpie  Pascal  environment  [HID-  SmaRtak 
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5.  Human  Factors 

Mnougn  mmmgjf  onvm  otvmopinifu  proouoM  pownu  sysiofns,  such  sysmrns  tsno  to  cow 
tain  numerous  *gadgets“  that  may  not  provide  the  most  effective  means  of  interaction  for  a  partic¬ 
ular  applcaion  or  uaar.  For  axampia,  manu  technology  for  command  invocation  may  increase 
the  efledveness  of  some  users,  yet  dacraaaa  lie  effectiveness  of  othar  uaara.  For  a  typist 
oonoamad  only  wMi  text  antry,  efficiency  ia  aavaraly  handtoappsd  I  certain  functions,  such  as 
typa  atyia  aalactlon.  can  only  ba  dona  by  manu.  Thu*  foe  match  among  technology,  appfication, 
and  user  is  critical  to  tha  overal  aflactivanass  of  the  system. 

The  term  ergonomics  has  boon  given  to  research  that  etudes  work  and  working  environments. 
Economics  atm  ease  tha  centrai  tola  of  human  physiology  in  foe  dasign  of  equipment  and 
woriaDlaoaa.  Tha  induda  daalon  fasauaa  of  lha  indMdual  wotkaoace.  tha  tvoa  of 

amktha  IndMdual  ia  to  perform.  and  tha  mathodt  by  which  tha  work  is  moat  affldandy 

SMCUtod. 
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Hufnsn  iigofi  •ngvwtnng  is  oocsif  rswso  vo  sfQonofivcs.  it  is  «  nso  ot  sngvissnng  pncuos 
mat  onsrs  ootn  pnenew  assign  gmossnss  sno  svssisDon  mstnoaoogMS  vw  nsp  computer 
dsstansn  wMh  thsir  oioducls.  I  toanst  on  Ihs  ussr  sspsets  of  hunttnaomulsf  Interaction.  It 

a  aa  aba  ^a  aaaw  avv^a  aa^a^aB  aaa  iaav^av  a^aawB^B^a^^^aa  aavranavaiva  va 
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avacey  appaaa  Knowraaga  ana  mamoas  tnai  nan  Dean  oanvao  worn  via  numan  acwncaa  auen 
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psycnoogy,  pnysioiogy i  sno  ms  sns*  ns  wononQ  unosssogs  is  snnenso  oy  ^ns  mots  aWOnns 
ano  suofewe  Rnowiaaga  or  practical  experience  from  paopia  wno  oaaqn,  ouso,  ana  uss  sys- 

toms. 


Card.  Moran,  and  Newel,  members  of  tha  Applad  information  rroraBBjng  rBychotopy  Projact  at 
Xarax  Parc,  wara  plonaara  in  applying  msthods  and  insights  from  cogntive  psychology  to  the 
practical  design  of  oomputara  [26].  Tha  range  of  human  psychology  actMNes  spans  perception, 
performance,  memory,  learning,  probiam  solving,  and  paychofinguiaHce.  Card  at  al.  envision  sub* 
mm  si  ms  yppacaaon  or  numsn  psycnoiogy  to  inionTiaDon  ptocsssng  suen  ss  ussr  scisnoss, 
cognMve  srgonomlca,  software  psychology,  cognitive  engineering,  and  artificial  peycholnguistics. 
Tha  Technology  Identification  and  Assessment  Proiect  draw  on  work  from  the  human  factors  area 
that  dsato  drectiy  wth  user  interfaces.  For  information  regardng  learning  and  problem  solving, 
references  are  given  for  the  relevant  fterature. 

A  variety  of  introductory  and  survey  materials  are  avaHabie  covering  various  aspects  of  human 
factors.  Burch  [17]  provides  a  recommended  reading  1st  on  computer  ergonomics  and  user 
Manly  design.  Rameey  [99],  Card  at  al.  [26],  and  Shneiderman  [115]  provide  extensive  reviews 
of  psychology  in  human  factors  engineering.  A  more  recent  Rterature  list  of  the  field  is  available 
in  tha  SIGCHI  BuMln(63]  and  as  part  of  an  article  by  Foley  et  al.  [42].  The  field  has  also 
roosived  attention  through  a  aeries  of  workshops  and  conferences  organized  by  various  profes¬ 
sional  organlzaMoni  such  as  ACM  SIGCHI,  tha  Human  Factors  Society,  and  the  IFIP  working 
group  (W. 


Tha  remainder  of  this  section  on  human  factors  is  structured  as  foftows.  First,  different  models  of 
Me  human  aspect  in  lie  interaction  process  are  dscussed.  Then,  methods  and  experimental 


Card  at  ai.  ado  developed  a  modal  to  study  the  Interaction  proceee  between  humans  and  com¬ 
puters.  The  model,  coled  GOM8,  consiets  of  goafs*  operators,  metfiods  for  achieving  the  goals, 
and  eodolbn/ude  tor  choosing  among  competing  methods.  They  use  this  model  to  analyze  the 
human-computer  interaction  process  tor  the  applcatlon  domain  of  text  edMng. 


Norman  (89]  proposes  a  sfenlar  dMsion  of  the  process  into  four  stapes:  Intention,  selection, 
execution,  and  evaluation.  Intention  reflects  a  set  of  pools  in  the  user's  mental  model.  Selection 
is  the  mental  eslecllon  of  actions  or  methods  tor  achfotong  the  goal.  Execution  is  the  physical  act 
of  entering  Information  into  the  computer.  Evaluation  Is  the  examination  of  the  feedback  as  part 
of  the  resuR  of  execution  to  determine  further  acdvty  In  the  process.  Norman  analyzes  the  four 
stages  to  determine  toe  need  ter  dWetsnt  support  at  dtaerenl  times  In  an  interactive  session.  He 
points  out  that  each  Interaction  technique  provides  a  set  of  trade-offs.  The  choice  among  the 
trade-offs  depends  on  the  technology  used,  the  class  of  users,  and  the  goals  of  the  design.  The 
choioe  has  to  be  made  wth  a  global  perspective,  because  the  optimal  choice  for  one  stage  may 
not  be  ootfmal  for  snnlher 

swro  ae^r  aegesM smbs  sees  sawwnpv* 


May's  work  (106]  elaborates  on  the  tatenUonal  stage  of  the  above  model.  Formulating  the  inten¬ 
tion  and  making  a  selection  requires  knowledge  of  the  actions  avaftabie  from  the  system,  and 
planning  for  achieving  the  decked  goals.  RBey  suggests  that  a  planning  framework  is  usefol  as  a 


Various  studtoe  hove  atasreptod  to  understand,  analyze,  and  measure  dtferent  aspects  of  human- 
computer  Maradoa  Buxton  (20]  dfooueeee  two  daaeWcatlcns  Into  which  those  studfoa  can  be 
cUSBOrtaadS,  ttiiPpinpeetien  isgwetnenitor  each  of  toe  rtasetcaltone.  Both  daseWcallons  were 
developed  horn  a  language-oriented  view  of  the  human-computer  dfotogua.  The  first  ciassin- 
caden,  MNBy  tabrntaoed  In  Foley  f4ll  tekes  a  language  view  Of  the  user  dWogue  and  proposes 
tour  foyers:  conceptual,  sementic,  syntectic,  and  textesL  The  second  dassMcation,  proposed  by 
Moran  (80),  tries  to  address  the  device  level  of  oommunlcabon  In  more  detaB.  It  divides  the 
problem  area  Into  a  ccnceptiuel  component  composed  of  a  task  level  and  a  sementic  level,  a 
oomm0ticstionooaoMtentOQn%iueed  tie  syntectic  tevel  trim  interaction  level,  arts  physic* 
component  mnqmM  of  a  foadbf  level  end  a  device  leveL  Buxton  proposes  a  refinement  of 
these  ctasallcaaone  by  addhg  the  pragmatic  level  to  better  address  spatial  issues  of  the  dia¬ 
logue. 

Card  at  «L  [W]  take  a  oogntovs  psychology  view  of  human-computer  interaction.  They  describe 
the  modal  of  a  human  prooseaor  as  consisting  of  a  perceptual  system,  a  motor  system,  and  a 
cognitive  system.  Human  performance  is  determined  by  perception  (readfog  rate),  motor  skll 
(key  layout  and  typing  efficiency),  staple  decision  making  (yeatao  answers),  and  foaming  and 
retrieval  (memory  of  aoquked  information). 
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naianar  [106q  reports  on  an  tmpirtcal  aludy  of  query  languages.  In  this  study  he  defines  a  meas¬ 
ure  of  lttmab%  and  teat  of-use.  Based  on  the  results  he  provtdea  feedback  for  query  language 
design.  to  adJUon,  toe  report  oomments  on  methods  for  evaluating  human  factors  and  tries  to 
arrow  me  reaoar  now  io  taerprer  tne  resuas  or  aucn  sajmst. 
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5.3.  Pictorial  Representations 

Tha  fom»  In  added  Information  tt  rapraaantad  strongly  affacta  tha  lima  8  takaa  lor  tha  human  mind 
to  prooaaa  R.  R  to  commonly  aboaffed  that  human  vlaual  orientation  aRowa  ua  to  piooaaa  vNuai 
InlonnaMon  In  dWawnf  waya.  Aa  naadar  (B6|  pobda  out,  vWon  aRowa  blatant  random  aooaaa  to 
any  part  of  a  pfdura.  provteRng  datiiad  and  omarNI  vtewa;  taxi,  however,  N  acannad  aequenfeRy. 
Pictorial  wpiaaanlaRon  prwtdee  more  dbnenafone  aooordbiQ  to  which  information  can  ba  laid  out 
i  vxk  m  DHcny  i  ons-osnsnsionsi  hmti  ov  cmractifi,  suQmsnisn  oy  POvmsuiQ  (suen  w 
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pictorial  language.  As  MoCleary  dtooueeee,  [76].  size,  value,  dfcscMon,  texture,  shape,  and  color 
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Into  graphical  form,  is  a  task  In  vrttoh  graphic  artists  and  ■ustrators  hava  extensive  experience. 
As  TuMa  shows  [13%  graphics  can  ba  an  effective  way  to  describe,  explore,  and  summarize  sets 
of  date.  However,  graphics  can  easly  be  misused  to  produce  what  he  cals  chart  junk.  Tufte  gives 
sows  guidelines  lor  exceSenl  statistical  graphics.  In  his  words,  graphics  can  be  more  precise  and 
reveaSng  than  etsMettcal  computation,  and  communicate  complex  ideas  dearly,  precisely,  and 
efficiently. 


.  Design  Principles 

design  principle*  summarized  in  this  section  have  several  sources:  authors  such  as  New- 
and  SprouS[88],  Foley  and  van  Dam  [41],  Good  [50],  and  those  summarized  in  Shneider- 
[1150;  a  questionnaire-based  study  [33];  and  the  cognttlven  psychology  realm  [26].  Despite 
Rversty  of  sources,  there  is  considerable  agreement  on  the  principles.  Some  apparent  corv 
tradkdons  among  principles  occur  because  the  criteria  summaries  do  not  dearly  dffierentiate 
between  classes  of  users.  Examples  Include  request  tor  redundancy  In  the  command  set.  and 
system  vs.  user  control  over  the  dtalogue.  The  design  criteria  are  grouped  as  folows: 

•  Know  the  user  —  adapt  to  the  user's  capabWties;  adapt  wortfness  of  both  input  and 
output  to  the  user's  needs;  alow  choice  of  entry  patterns;  provide  shortcuts  for 
knowledgeable  user  provide  guidance  when  desired;  use  the  user's  model  for  the 
problem  domain. 

•  Consider  personal  worth  of  die  user  —  leave  action  initiation  raid  control  (e.g.,  abor¬ 
tion  of  command)  to  the  user,  allow  the  user  to  decide  on  dialogue  technique  and 
talor  the  dialogue  pattern;  give  quick  and  meaningful  feedback  on  state  and 
progress  of  interaction  and  execution;  do  not  require  the  user  to  supply  Information 
that  is  already  avalabie;  maintain  a  well-balanced  social  element  In  the  dialogue. 

•  Reduce  learning  —  provide  consistency  and  dear  wordtog;  reduce  modaffiy;  main¬ 
tain  uniformity  and  homogeneity;  avoid  overioadng  of  commands;  provide  self- 
explanatory  commands  and  help  when  requested  or  needed. 

•  Engineer  tor  errors  —  provide  consistency  and  uniformity;  handle  all  possfcie  input 
combinations;  provide  good  error  messages;  eflminate  common  errors;  provide  pro¬ 
tection  from  costly  errors;  provide  support  tor  correction  of  errors;  provide  context- 
sensMve  onfine  assistance. 

•  Know  the  application  —  present  a  natural  image  of  the  application  system;  provide 
problem-oriented  powerful  commands;  carry  forward  a  representation  of  the  user's 
knowledge  base  about  the  application's  problem  domain. 

Card  et  al.  [26]  and  Buxton  [18]  consider  the  time  component  (the  expected  performance 
requirements)  to  be  an  addkional  design  criteria.  Card  et  al.  further  propagate  a  set  of  principal 
steps  that  should  be  folowed  to  achieve  good  user  interface  design: 

1.  Set  goals  to  be  met  (performance  requirements,  target  user  population,  problem 
and  user-oriented  definition  of  tasks), 
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6.  Implementation  Considerations 

Thto  section  dtocuases  user  interfeces  from  the  view  of  the  application  implementor,  who  can  be 
faced  wkh  quite  a  dRferent  interface  from  the  user  interface  system.  A  user  interface  system  may 
be  easy  to  use,  but  hard  to  interface  wth  an  application  program.  The  interface  may  provide 
Rmfted  functionally,  funettonafty  not  appropriate  for  the  applcation,  or  a  certain  interface  style, 
which  would  not  alow  the  implementor  to  specify  his  own  style.  The  folowing  examines  how 
application  user  interfaces  have  been  implemented 

In  the  simplest  form,  the  applcation  programmer  uses  I/O  routines  provided  by  the  programming 
language  or  by  a  subroutine  fcrary.  Often  these  facades  hands  specific  input  and  output  devices 
or  device  classes  (separate  routines  for  terminal  and  ffle  I/O)  and  dRferent  caMng  parameters  for 
dHerent  terminals.  This  resuRs  in  device-dependent  applcations  that  are  dfficul  to  port  Device¬ 
independent  I/O  models  moke  applcations  mors  portable.  Examples  of  such  models  include 
uniform  file  and  device  I/O  in  Unix,  virtual  device  interfaces  such  as  the  Unix  curses  and  termcap 
faefflty,  and  standard  ferary  routines  for  input  processing  such  as  the  Unix  fcraiy  routines  for  text 
input  handling.  Increases  in  sophistication  of  user  interface  functionality  results  in  more  complex 
software.  Since  a  considerable  part  of  an  application  represents  user  interface  code,  and  en¬ 
hancements  to  software,  there  is  a  desire  to  localze  the  user  interface  code,  to  share  and  reuse 
the  code  across  applications,  and  provide  higher-level  functionary  by  predefined  user  interface 
packages. 

The  term  user  interface  management  system  (UIMS)  has  been  coined  (66, 132]  tor  tools  that 
assist  in  the  development  of  good  Interactive  application  systems.  The  applcation  developer  ex¬ 
pects  benefits  from  a  UIMS  [18, 125, 58]  such  as: 

•  device  independence  of  the  application, 

•  reduction  in  programming  effort, 

•  reuse  for  several  applcations  resulting  in  mors  consistency,  and 

•  quicker  and  mors  effective  prototyping  and  tailoring  of  the  user  interface  of  an  appli¬ 
cation. 

The  UIMS  is  expected  to  provide  functionality  at  the  appropriate  level  to  model  effectively  the 
user's  view  of  the  applcation  without  imposing  preconceived  policies  on  the  form  of  dialogue. 
Further,  R  to  expected  to  be  decoupled  from  the  applcation  to  permR  tailoring  of  the  dialogue 
without  affecting  the  applcation  part,  whle  stll  maintaining  a  correct  and  consistent  display  of  the 
application  and  Its  state. 

The  folowing  section  dtocusses  arohRecturai  models  of  user  interface  software,  examines  tools 
for  developing  user  interfaces,  and  elaborates  on  the  desire  for  portabWty  and  the  necessary 
standardization  efforts. 
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The  interaction  between  the  aoolcation  and  tha  uaar  interface  it  detcribed  in  a  teoond  model 
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the  control  areMaduro  modal.  Three  Idndi  of  control  archftedure  can  be  Identified:  Internal, 
external,  and  mbced[\25\. 

•  In  the  internal  control  modal,  the  appkeaton  it  in  charge  of  the  flow  of  control.  By 
caBng  on  functions  in  the  user  interface,  ft  determines  the  flow  of  the  dtalogue. 

•  The  external  control  model  tafcee  the  inverse  view,  and  the  appftcation  Is  provided  as 
a  sat  of  functional  unfta,  which  tha  uaar  interface  system  invokes  in  response  to  user 
intacacbons.  Hence  tha  uasr  or  user  intsrface  Is  in  control  of  the  dialogue  sequence. 

•  The  mixed  oontroi  model  supports  concurrent  execution  of  the  appftcation  and  the 
user  interface  in  separate  processes.  They  communicate  through  a  message  sys¬ 
tem.  The  user  interface  system  must  hancit  two  asynchronous  interfaces:  the  end 
user  and  the  appftcation.  8uch  a  modal  provides  mom  flextottty  in  that  ft  can  deal 
wlh  uaar  or  appftcatfon-oenoratod  events  that  art  not  part  of  the  main  flow  of  da- 
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The  composMon  of  an  appHcatton  dWbgue  from  avalabie  bulking  blocks  can  be  formaly  speci¬ 
fied.  In  a  language-oriented  view  [42, 20],  several  aspects  of  the  interaction  dialogue  have  to  be 
considered:  pragmatics,  lexical  elements,  syntax,  and  semantics.  Different  formalisms  are  most 
appropriate  for  each  of  these  aspects.  This  architectural  model  has  some  shortcomings,  in  that  it 
does  not  dearly  address  issues  of  two-<*mensJonal  and  graphical  cfispiay,  and  issues  regarding 
the  interaction  between  user  input  and  feedback  of  the  system.  Shaw  et  al.  [113]  refer  to  the 
capturing  of  this  interplay  between  input  and  output  as  prosody.  They  point  out  that  attention 
must  be  paid  to  the  way  decisions  are  made  about  format  and  prosody.  By  localizing  this  infor¬ 
mation,  poNcy  decisions  regarding  the  layout  can  be  defined  in  a  database,  and  style  definitions 
can  provide  initial  choices  (an  idea  already  present  In  the  Scribe  text  formatting  system  [102]). 

Shaw  [1 1 4]  presents  a  new  model  for  input/output  that  overcomes  the  deficiencies  of  the  classical 
language-oriented  model.  Her  model  suggests  a  software  organization  that  supports  handling  of 
application-defined  data  abstractions,  composition  and  maintenance  of  a  display  image,  and 
processing  of  appflcation  and  user  events. 

6.2.  Development  Tools 

Tools  for  developing  user  interface  software  come  in  two  categories:  software  building  blocks 
and  languages.  Bulding  blocks  are  made  available  tot  the  form  of  tooboxes  or  in  the  form  of 
generation  systems.  Languages  exist  in  the  form  of  programming  languages  for  specification  of 
the  interface  between  the  appNcation  and  the  user  interface,  and  in  the  form  of  specification 
languages  for  user  interface  designers  to  define  the  appearance  and  behavior  of  the  user  inter¬ 
face.  Each  of  these  categories  wll  be  treated  in  turn  in  the  next  two  sections. 

6.2.1.  Toolboxes  and  Generation  Systems 

Tanner  and  Buxton  [125]  discuss  an  evolving  model  of  user  interface  management,  i.e.,  the  com¬ 
bination  of  the  process  of  building  a  user  interface  and  the  runtime  architecture  of  the  user  inter¬ 
face.  They  introduce  the  terms  module  builders  and  glue  systems  to  refer  to  tooboxes  and  gener¬ 
ation  systems. 

Tooboxes  provide  a  Ibrary  of  building  blocks  from  which  a  user  interlace  can  be  manufactured. 
The  user  interface  builder  may  be  provided  with  tools  to  enhance  the  building  block  set.  e  g.,  icon 
and  font  editor  to  add  application  specific  symbols.  The  building  blocks  may  provide  different 
levels  of  functionality.  The  Macintosh  Toolbox  [5]  and  Microsoft  Windows  [70]  are  two  examples 
of  tooboxes  from  which  the  builder  will  pick  building  blocks  and  assemble  them  to  form  the 
preferred  user  interface.  Certain  aspects  of  foe  interface  are  predefined,  e.g.,  the  placement  of  a 
menu;  others  are  in  the  control  of  the  bulkier.  In  contrast,  Andrew  [110]  is  an  example  of  a  toolbox 
that  provides  a  very  high  level  interface  and  imposes  built-in  policies  for  the  user  interface  ap¬ 
pearance  and  behavior. 

User  interface  generation  systems  aOow  foe  builder  to  formally  specify  foe  appearance  and  be¬ 
havior  of  the  user  interface.  This  specification  is  then  processed;  the  use  of  appropriate  building 
blocks  to  achieve  foe  effect  is  determined;  and  the  information  is  deposited  in  a  knowledge  base. 
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The  rurtkm  aupfHMK  of  toe  system  executes  the  user  interface  by  Interpreting  this  knowledge 
haeeanrl  executing  toe  toraiy  routines.  Examples  of  such  genswUon  systems  ere  products  such 
as  ApoloKMogH).  and  leeeuoh  systems  such  as  Tigsr[6fll  and  Cousin  [581.  Some  generation 
Mieim  dMar  In  that  tte  user  Interface  is  not  eoecftod  bv  a  textual  description,  but  throuoh  an 
Meracttvs  loci  (e.g.,  Flair  [141]  and  Menubqr  pi]).  The  Msractive  specification  of  the  user  inter- 
not  vna  mv  anci  viwpcinBon  or  *nv  KnowivaQv  dsm  mm  mt  construction  ana  mow  tor  rapta 
ppMoq^anp.  uttonpoom  ana  oofnpmons  01  ssvsrvi  user  nanaca  vnanaQavnavw  systems  can 
be  found  In  Buxton  et  at  pi]  and  Olsen  et  aL  {92]. 


Kis  Merssdno  to  note  that  tyeaw-^tflt**  edbor  systems  have  oone  throuoh  a  simBar  evolution. 


Thera  are  handcrafted  systems  [127],  systems  generated  from  formal  descriptions  [77],  and  sys- 
nmsiMoriaio  r^ta  pvoi^vypviQ  oy  pcovong  nincm  ytcwctton  ana  avsct  vvorpreiaDon  or 
thatamaOSl  Hay  nan  ha  oonsiclirsfl  utif  Inisrt  set  manaoamani  syntims  1381  with 

(•  llbaA  Mi^a  aiaiia&j  afev  imA  aMiteAia  SiMMiSLkMaBhj  Lka  MAAhlAatkpA  apu4  Mamd 

vi  mat  may  uauay  oo  not  pnaat  TunctJOnMcy  vor  anxrary  poMKywnp  ana  gnpnicai  output. 

Issues  to  be  addrseeed  ere  the  separation  of  oolcv  and  mechanism  M 131.  the  inclusion  of  deskm 
ouidslnes  es  e  knowtsdos  boso  Into  the  user  Mftort  osnsratlon  system.  and  the  abMv  to 
Include  new  technotoglee  end  techniques  such  as  spssch  Input  [141]  and  odor  into  user  interface 
sysesms.  ina  stfMVMon  01  vnaonareavn  ana  poaoy  va  anaa  aovna  aym  on  ma  appvopnaia 
paramacanonon  fuaar  mdviw  a  via  uaar  navn.  Lmosewny  otagn  gutavas  no  tne 

MAakApMAtAjk  pBjBApPV  pp^p  ilapp  pp  pj  Ikp^p  p.  al^plMAA  ^4apSpIpP  M^aapI  AM  tkppp 

ganamon  tynani  rsquws  a  vonvvnMi  ot  maaa  gunaanaa  ana  pnncsMS.  Baaaa  on  vast 
guklalnes  and  mtoroprlste  Input  from  toe  appfcation  designer,  toe  eyetam  generates  a  user  Inter- 
face.  in  the  area  of  buainne  frfr^Tft  an  atiemot  has  bean  made  to  oaulure  such  information 
into  toe  system  [47].  Rather  lhan  toe  ueer  eeiectinQ  prseentatlon  mechenisiiis,  suoh  as  piechsrts, 
histograms,  and  choioe  of  color,  the  ueerspecllee  to  the  system  the  effect  to  be  achieved  by  the 
nmeenfUnn;  that  Is.  whether  e  trend  analvaie  is  Mended  or  the  data  ie  to  be  oomoared  in 
absolute  or  rotative  terms.  Baaed  on  toe  answers  and  the  type  d  data,  toe  system  automatical? 
determines  an  appropriate  presentation. 

6J3L2.  Languages 

Language  euppon  Ten  no  iwo  caiegonee.  pcogwnnwig  languages  ana  epecmcaaon  languages. 
Programming  languages  have  been  extended  to  provide  more  lhan  toe  classical  support  [1 14]. 
Examples  d  such  work  are  PS-Algd  [81],  extension  d  Pascal  [68],  Screen  fhgd  [100],  and  Taxis 
[96].  Extensions  include  predefined  types  that  are  apecMc  to  the  ueer  Interface  and  language 
constructs  for  carrying  out  user  interface  operations.  The  aftemative  to  language  extensions  is 
the  provision  of  a  subroutine  Nbrary.  Work  has  alto  been  done  to  define  interfaces  to  graphics 
systems  in  a  language  Independent  manner  [120],  and  to  automatically  generate  language- 
apedfic  interfaces  and  the  necessary  support  routines  tor  massage-oriented  procedure  cads 
[100, 4]. 

A  range  of  formal  specification  languages  have  been  employed  In  specifying  the  appearance  and 
behavior  of  user  Interfaces.  Some  we  applications  of  existing  notations  with  extensions;  others 
are  special  purpose  languages.  Their  purpose  it  to  capture  the  syntactic  nature,  the  format,  and 
the  control  flow.  Omen's  thesis  [53]  surveys  Offered  graph-based,  grammar-based,  and  al¬ 
gebraic  spsdflcation  techniques.  Extended  forms  of  the  Backus-Naur  Form  [13, 91, 56]  and 
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variattona  132. 771  haw  ban  uaed  State  traneiion  <»  «nrarw  nn  auamantad  tranaion  networks 
hava  baan  employed  {23, 62, 136, 64].  Petrinets  and  other  graphical  specification  techniques  are 
attraettoe  candtoeiai  Chi[28|  oomparaa  and  evatoatos  four  axiomatic  approaches  tor  formaly 
spedfylngtoesemardcsof  uaer  interfaces.  Base  [11]  dtocueaee  a  toeory  of  grouping  and  a  sys- 
tain  that  aupporta  formal  uaar  specMcaUon  of  form-based  uaer  interfaces  acconAng  to  this  theory. 
Special  DlirDoaa  lancaiaaaa  are  tafiorsd  to  the  naada  of  the  apolicaPon  area  auch  aa  iancuaaea 
tor  deacrlption  of  office  intormatton  environments  [34],  or  languages  fy  specifying  picturea  [135]. 
Languages  have  been  designed  that  are  based  on  state  transftions,  yet  include  other  aspects 
such  aa  layout  aa  wel  [53, 1 13].  Hnaly,  languages  exist  tor  defining  data  representations  and 
views  on  them  in  databases  [1],  syntax  erfltors  [77],  and  language  systems  [87], 

6.3.  Portability  and  Standards 

CompatibiSty  of  systems  is  a  desirable  attribute  because  I  permit  integration  and  porting  with 
Bttle  or  no  cost  Since  software  coats  art  a  major  factor  in  system  development,  oompaNbMy  is 
important  to  user  interface  technology.  Compatibly  to  achieved  through  common  agreement  to 
pubic  interfaces.  This  can  happen  intormaly  by  a  large  population  accepting  someone’s  interface 
fdefacto  standards),  or  bv  an  B»iMw*aM  rinnimartlnri  nnhiahinn  and  oremotina  an  interface  atari* 
daixL  Organizations  such  as  ISO.  CCfTT.  MBS.  ANSI,  IEEE,  ACM,  and  the  DoD  are  active  in 
promoting  standards  related  to  user  interface  technology.  The  remainder  of  this  section  dto- 
cusses  different  dknensions  of  portabMy  and  NgMghts  some  of  the  official  and  defacto  stan¬ 
dards. 

6.3.1.  Portability 

PortabMy  can  be  viewed  from  many  angles:  portabMy  of  users  and  of  software:  or  portability 
across  different  operating  systems,  devices,  machines,  and  languages.  Users  portabiHty  is  a 
prominent  aspect  in  user  interface  technology.  As  the  interaction  style  and  the  command  lan¬ 
guage  differ  from  appfcaMon  to  applcation  and  system  to  system,  users  are  constantly  in  the 
process  of  learning  new  ways  of  communication  (tower  of  Babel).  This  process  can  be  made 
easier  I  uniformity  and  consistency  exist  Through  reuse,  user  interface  management  systems 
can  contribute  by  providtog  a  consistent  set  of  vocabulary  and  gestures.  This,  however,  contacts 
with  the  user  interface  designer's  attempt  to  present  a  conceptual  view  of  the  application  that  is 
dose  to  the  user's  mental  model  of  the  application  and  makes  the  interaction  more  natural  and 
leamabie. 


PortabMy  of  languages  has  two  aspects.  One  aspect  to  support  for  an  interface  between  the  user 
interface  system  and  toe  applcation  such  that  applications  can  be  written  in  (Afferent  program¬ 
ming  languages.  MuUpie  language  support  can  be  provided  in  several  ways.  One  approach  is 
taken  by  VAX/VMS  and  Apoflo'DOMAJN  where  al  languages  share  a  runtime  environment,  and 
interianguage  cals  are  supported.  Limitations,  however,  may  exist  in  some  languages  in  toe 
support  of  data  abstraction  and  event  processing.  Another  approach  makes  use  of  toe  property 
that  the  application  and  toe  user  interface  run  in  separate  processes.  The  message  protocol  or 
remote  procedure  cal  mechanism,  if  defined  appropriately,  can  interconnect  processes  running 
(Afferent  language  systems  [101].  A  second  aspect  of  language  portability  is  support  for  multiple 
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natural  languages.  As  shown  in  tha  Macintosh  product,  al  cSaiogue  text  can  be  kept  in  a  data¬ 
base  separate  from  the  appication  code.  However,  the  situation  has  to  be  considered  for  several 
people  wth  dMerent  natural  languages  using  the  same  machine. 

User  interfaces  and  appMcatbns  are  usualy  dependent  on  the  operating  system.  For  example, 
the  Macintosh  Tooldt  and  Microsoft's  Windows  were  designed  for  and  implemented  on  single 
process  operating  systems.  Design  decisions  inNM  those  user  interface  systems  from  being 
easfly  made  avalable  on  other  operating  systems.  MuMpie  process  architectures  for  user  inter¬ 
faces  make  use  of  message  systems  and  remote  procedure  cal  mechanisms. 

PortabMy  across  input  and  output  devices  has  been  handed  by  defining  virtual  interfaces. 
Device-specific  knowledge  is  hidden  in  the  implementation  of  the  interface.  Examples  are  the 
OKS  logical  devices  for  graphicaVblmap  displays,  and  the  Unix  fBe  I/O  interface  and 
curses/lermcap  package  for  character  terminals.  By  buitdng  on  such  virtual  interfaces,  large  por¬ 
tions  of  the  user  interface  system  can  be  kept  device  independent  This  ger<eral  statement  is  true 
for  devices  within  a  device  class.  However,  the  functionally  of  devices  in  one  class  may  not  be 
sufficient  to  simulate  the  functionality  of  devices  in  another  class.  Therefore,  the  user  interface 
design  may  have  to  be  adapted  for  dfferant  device  classes.  Wkh  an  appropriate  user  interface 
management  system,  this  information  can  be  locabed  in  the  user  interface  specification.  The 
application  does  not  have  to  be  affected. 

Porting  of  appfcations  to  other  machines  includes  adaptation  of  the  user  interface.  This  can  be 
aooompSshed  in  two  ways:  by  keeping  the  user  interface  system  portable;  or  by  defining  a  stan¬ 
dard  Interface  between  the  application  and  the  user  interface  that  is  implemented  by  dlferent 
user  interface  systems  on  dNferant  machines,  in  a  network  environment,  a  third  atematlve  is 
poesble.  Access  to  an  appBcation  from  another  machine  can  be  provided  without  a  port  of  both 
user  interface  and  appication.  In  the  mixed  control  model,  the  user  interface  and  the  application 
are  partitioned  into  separate  processes.  Network-wide  interprocess  communication  or  remote 
procedure  cal  sendee  makes  a  dtetrbuted  setup  possbie.  Other  partlions,  such  as  at  a  virtual 
device  level,  can  also  be  envisioned.  The  netwoik  bandwidth  wil  determine  the  effective 
feasibility  of  dMerent  distribution  scenarios. 

An  interesting  special  case  of  porting  application  between  devices  or  user  interface  systems  is 
the  interconnection  of  applications,  i.e.,  driving  one  application  from  another.  Most  interactive 
applications  cannot  be  executed  from  another  program,  whether  user  program  or  batch  proc¬ 
essing  program,  due  to  restrictions  in  the  operating  and  runtime  system.  Even  on  operating  sys¬ 
tems  that  encourage  couplng  of  appfications,  such  as  Unix,  this  is  not  always  possbie  because 
the  appication  has  embedded  knowledge  about  conversing  with  an  interactive  device. 

6A2.  Official  Standards 

Official  standards  are  most  prevalent  at  the  device  level.  Only  a  few  efforts  exist  at  more  abstract 
levels  of  the  user  interface. 

Character  sets  are  encoded  in  ANSI’s  ASCII  standard  or  in  defacto  standards  such  as  IBM’s 
EBCDIC.  The  ASCII  standard  exists  for  7-bit  and  8 -bit  encoring  as  well  as  for  a  multilingual 
graphic  character  set. 
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ANSI  has  defined  a  standard  (X3.64)  for  the  interface  between  CRT  terminals  and  computers. 
This  standard  is  very  dose  to  the  functionaKty  of  the  DEC  VT100  terminal.  Other  terminal 
hardware  vendors  have  adopted  this  standard  by  providing  an  emulation  mode.  (As  it  turns  out,  It 
to  the  VT100  terminal  more  often  than  the  X3.64  standard  that  is  being  emulated.)  On  worksta¬ 
tions  with  bitmap  displays,  window  systems  provide  one  window  type  to  emulate  this  terminal 
standard  (97J.  Such  an  emulation  permits  the  workstation  to  be  treated  as  a  terminal  to  another 
machine  and  allows  programs  to  be  ported  and  run  on  the  workstation  with  little  effort,  but  without 
adaptation  to  addttional  capabtoties  of  the  window  system. 

Several  standards  exist  in  the  graphics  area.  Close  to  the  hardware  level,  a  standard  for  a  device¬ 
independent  interface  to  different  graphics  equipment  was  defined  in  the  form  of  a  virtual  device 
Interface/metafBe  (VDI/VDM),  now  known  as  computer  graphics  interface/metafile  under  ANSI 
X3H3.3.  The  metafle  specification  defines  a  mechanism  for  storing  and  transmitting  graphical 
images.  Below  the  VDI  level,  NAPLPS  defines  a  standard  method  for  encoding  character  and 
point  information  for  driving  videotext  devices.  The  OKS  defines  a  virtual  graphics  model  and 
provides  an  interface  to  the  appNcaUon.  A  device-independent  implementation  of  GKS  resides  on 
top  of  the  VDI  level.  Bono  [14]  explains  the  interrelationship  of  these  and  other  graphic  standards 
in  more  detaH  and  dtecusses  their  role  in  information  interchange. 

GKS  was  adopted  as  a  standard  by  ISCVTC  97  (ISO/DIS  7942).  The  standard  indudes  proposals 
for  language  bindngs  of  the  GKS  package  to  common  languages  such  as  Ada  (ISO/DP  8651/3). 
Proposals  exist  for  a  GKS  for  three  dknensions  (ISO/DP  8805),  and  a  metafle  description  for 
graphical  representations  (ISCVDP  8632):  Waggoner  [136]  points  out  cfifferences  in  the  two  stan¬ 
dards  and  gives  reasons  for  the  success  of  GKS.  Developing  UIMS  on  top  of  GKS  has  proven 
feasble,  but  has  also  uncovered  some  shortcomings  due  to  the  mismatch  between  the  model  of 
the  graphic  system  and  the  UIMS  [132].  The  tools  avalabie  in  the  standard  graphic  system  are 
inadequate  for  supporting  static  or  dynamic  division  of  the  screen  into  windows  [106]  and  for 
providing  a  linkage  between  input  and  output  [92]. 

Standardization  efforts  are  in  progress  for  document  interchange  formats.  ISO  has  proposals  for 
standardization  of  text  preparation  and  interchange  for  text  structures  (ISCVDP  8613/1-6)  and  for 
processing  and  mark-up  languages  (ISO/DP  8879).  For  a  status  report  on  this  and  other  efforts  in 
document  interchange,  the  reader  to  referred  to  Horak  [59]. 

Military  guidelines  and  advisories  exist  in  various  forms  such  as  MIL-STD-1472C  (Human  Engi¬ 
neering  Design  Criteria  tor  Military  Systems,  Equipment,  and  Facilities)  [31]  and  ESD-TR-83-122 
(Design  Guidelines  for  the  User  Interface  to  Computer-Based  Information  Systems)  [118]. 

6.3.3.  Defacto  Standards 

Unix,  which  has  two  major  branches  of  evolution  (Berkeley  4.2  and  AT&T  System  V),  to  becoming 
a  defacto  industry  standard  for  operating  systems.  Unix  has  made  several  advances  regarding 
user  interfaces.  It  provides  a  command  language  {ah  and  its  variants  csh  and  ksh)  that  enables 
programmabflity  and  easy  extensfoity  of  the  command  set.  It  supports  command  history,  execu¬ 
tion  of  previous  commands,  and  setting  of  commands.  Through  the  termcap  interface,  Unix 
provides  terminal-independent  access  to  CRT  terminate.  The  uniform  treatment  of  terminal,  file, 


7.  Summary  of  Issues 

The  hunt  of  Ms  Motion  It  to  summarize  Imum  regardng  afferent  aspects  of  user  interfaces. 
The  resolution  of  tht  tummartetd  Imum  it  viewed  as  advandno  the  state-of-pracdce  of  user 
Inlerfaoe  technology  end  hcrMthg  the  um  of  advanced  user  interface  technology  h  the  work 
environment  of  software  engineering.  The  iecuM  fal  Into  two  groupe.  The  first  group  deals  with 
mt  fsci  uai  ons  Gswiov  sxpso  m#  hsIQ  nOvnOQOnNus  oOiiipuiinQ  envwonrmro,  out  rnuSt  expea  10 
cope  with  heterogeneous  computer  equipment.  The  second  group  focuses  on  the  heterogeneity 
of  appfcaftont  (eoftware  tools)  and  users,  and  the  abtty  of  the  user  Interface  to  appropriately 
support  ft. 

7.1.  Heterogeneous  Computing  Environments 

Today  them  is  a  prevention  of  workstations.  These  workstations  can  tie  placed  in  one  environ¬ 
ment  by  wiring  them  together  into  one  local  area  network.  Furthermore,  users  may  work  from 
home  using  a  modem  and  a  character  terminal  or  workstation.  This  physical  connection,  how¬ 
ever.  does  not  make  the  reaouroM  avaftable  to  ttie  uaer  h  a  transcalent  manner  Aooflcattons  are 
bound  to  specVic  hardwam  and  cannot  be  executed  remotely  on  afferent  hardware.  The  dteplay 
functionally  provided  on  one  workstation  may  not  be  mralabie  on  anodier  one.  The  connection 
bandwidth  between  afferent  workstations  may  make  certain  mortes  a  operation  intolerable.  Is- 

mIW  01  w/aiSni  pWOnnlQi  pOvwN^fi  snu  9StnDUu0u  aMteKJ  10  DO  OQOTOOSOu  fti  OluOf  fO  DfiviQ 
Improvements  to  the  work  environment  of  eoftware  engineers,  most  Ifcely  heterogeneous  comput¬ 
ing  environments. 

Many  tools,  both  interactive  and  noninteractive,  make  up  the  work  environment  of  software  eng}- 
noiri.  9onwvo  oovonpmoni  onvionmofi  DUMOft  cormoi  m  oxpocioo  10  pfovioo  ai  foots  tnonv 
selves.  Tools  are  being  developed  by  afferent  groups  on  afferent  hardware  and  software,  provid¬ 
ing  dHerent  user  interfaces.  Integrating  this  mu*ipSc*y  of  tools  into  a  consistent  and  uniform 
environment  is  a  chalenging  task.  Prevention  at  hardware,  at  least  on  the  desk  at  a  single 
software  engineer,  must  be  reduced.  Toole  must  be  able  to  Interact  with  each  other,  share  and 
exchange  Information.  Furthermore,  I  is  desirable  for  a  user  to  Interact  with  one  conceptually 
uniform  user  Interface,  even  for  tools  on  different  hardware  and  operating  systems.  Existing  tools 
cannot  always  be  dscarded  and  must  be  adapted  to  the  user  interface  of  the  environment.  The 
increase  in  sophistication  at  user  interface  technology  and  the  lack  at  Interface  standardization  at 
a  high  functional  level  has  made  the  resolution  of  these  problems  a  difficult  task. 

7.2.  Heterogeneous  User  Groups 

Users  are  a  heterogeneous  population.  For  users  to  be  most  effective,  the  user  interface  of  a 
system  should  adapt  to  the  needs  and  capabffltles  of  a  user.  Preconceived  notions  of  expertise  in 
a  system  are  often  more  of  a  hindrance  than  a  hep  because  the  degree  of  expertise  can  vary 
greatly  even  wfthin  one  appfcation  and  can  change  over  time  by  nonuse  of  the  system.  Given 
appropriate  tools,  the  user  can  perform  some  tailoring  of  the  user  interface  himself. 
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ioosss  of  buMno  and  taRoring  user 
designer's  view  of  user  interactions, 
ride  user  interaction  sendees  and  a 
iganflng  appearance  and  behavior  of 
mn  design  principles  and  guidelines 
•doe  should  be  formalized  and  em- 


Thers  Is  a  tension  between  the  desire  to  taler  to  indMdual  users  and  tools  and  the  attempt  to 
achitvs  lie  appearance  of  an  Integrated  software  development  environment  with  a  uniform  user 
Wetfaoe.  At  issue  is  lie  appropriate  batonoe  between  flextottty  and  standardbration  of  the  user 
vnnaci  it  I  toRMwv  otwnpmmi  mvownini.  winwi  iDMQf  snoun  m  provKJM  oy  ma  m 
butt  Into  the  user  interface,  e.g.,  support  for  dMerent  degrees  of  expertise.  Certain  properties  of 
the  user  totfitaosahorid  be  epscltodonoofy  a  parloular  environment  and  left  rstotlveiystabie. 
W  parameterization  to  IndMduate  is  supported,  there  should  be  a  simple  way  for  changing  l  to  tha 
of  a  dtterent  indMdUai  in  short  notice,  e-g.,  one  person  waNng  up  to  another  person’s  dtepiay 
mo  Diwy  anp  ray  ran  raiiuig  oraiQOoraonrai  wmiraorai  rayowiOwiBS* 

The  tschnkiuse  for  evalualion  of  user  interfaces  rsaufes  further  attention.  Work  derived  from 
appaed  oognttve  psychology  reeuled  In  soma  models  to r  quanttattvety  measuring  basic  Mer- 
aetton  steps.  Adttttonai  models  that  address  other  aspects  of  user  interfaces  need  to  be  devei- 
oped  together  with  tools  for  carrying  out  lie  evakiatton.  Benchmarks  must  evolve  to  alow  for  a 
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■ogrephy  on  interactive  dtopiay  interfaces 
department,  Camsglo  Melon  University, 
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